Information Theory in Probability, Statistics, Learning, and Neural Nets (Abstract)

Probabilistic information theory determines fundamental limits in the topics of data compression channel capacity thermodynamics statistical estimation prediction hypothesis testing and related topics Moreover information theory provides an illuminating perspective on the limit the orems of probability Beginning with some identities and inequalities for relative entropy followed by their use in solving some problems in proba bility and statistics and concluding with application to neural networks this talk reviews the role of information theory in answering interesting questions in these topics